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This study addresses the need for reliable and valid information concerning how innovative 
classrooms on college and university campuses affect teaching and learning. The Social 
Context and Learning Environments (SCALE) survey was developed though a three-stage 
process involving approximately 1300 college students. Exploratory and confirmatory factor 
analyses supported a four-factor solution that measures formal and informal aspects of 
student-to-student as well as student-to-instructor classroom relations. The resulting 26-item 
instrument can be used by instructors to measure classroom social context in different types 
of learning spaces and to guide efforts to improve student outcomes. 


Introduction 

Recent years have seen rapid growth in the design, 
development, construction, and use of new-style classrooms 
at institutions of higher education in North America and 
beyond (Oblinger, 2006; Finkelstein et al. 2010). This 
increased interest is evident in major classroom redesign 
projects at leading universities (e.g., the Mosaic Initiative at 
Indiana University, https://uits.iu.edu/mosaic; the Team- 
Based Learning Project at UMass Amherst, 
http://innovate.umass.edu/team-based-learning- 
classrooms/; the Active Learning Classrooms project at the 
University of Minnesota, https://cei.umn.edu/support- 
services/tutorials/active-leaming-classrooms ), and in the 
fact that active learning classrooms (ALCs) were named the 
top strategic technology of 2017 by the EDUCAUSE Center 
for Analysis and Research 

( http://er.edu cause.edu/blogs/2017/3/active-leaming- 

classrooms-the-top-strategic-technology-tor-2017) . 

In parallel with the development of new learning spaces, 
a growing body of research has emerged that examines the 
changes brought about in the teaching-learning process by 
holding classes in these new classrooms. Some studies have 
found that, compared to near-identical classes taught in 
traditional classrooms, teaching a class in an active learning 
classroom yields improvements in both student affective 
responses and in student learning outcomes 
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(Brooks, 2010; Walker, Brooks & Baepler, 2011; Baepler, 
Walker & Driessen, 2014); other studies have yielded null 
results (Stoltzfus & Libarkin, 2016). What has been studied 
less often than the outcomes associated with different types 
of learning spaces are the possible mechanisms that underlie 
those outcomes. If new classrooms do affect teaching and 
learning, by what means do they do so? 

The purpose of this study is to contribute to answering 
this question by conceptualizing how learning spaces shape 
the teaching-learning process, focusing in particular on 
changes to the social aspects of learning. From a theoretical 
point of view, the contribution of this study is to break down 
the social aspects of classroom learning into clear, distinct 
dimensions as well as to provide a reliable and valid way of 
measuring those dimensions. Practically speaking, this 
study validates a simple self-report instrument that can be 
used by instructors to gauge the social components of a class 
and to guide efforts to promote the most educationally 
constructive of those components. 

Literature Review 

We came to the idea of social context through our early 
research on ALCs (Whiteside, Brooks, & Walker, 2010). In a 
series of focus group interviews, we asked instructors and 
students who had taught or taken classes in ALCs what was 
different about these learning, spaces; a common theme was 
that ALCs changed the social aspects of class, primarily 
through their physical layout. That the ALCs had no single 
focal point to draw attention seemed to influence instructors 
and students so that they interacted with each other more 
frequently than in a lecture hall - from instructors threading 
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through the student tables asking and answering questions 
to students facing one another and conversing. 

The idea that social context may be an important 
component of the educational process is bolstered by the fact 
that it has received substantial attention in the area of 
educational theory focusing on interpersonal relationships 
in the classroom (Amedeo, Golledge & Stimson, 2009; 
Meyers, 2008; Tiberius & Billson, 1991). 

Conceptually, social context has affinities with related 
concepts such as academic engagement, but it is not the same 
as engagement. In an educational context, engagement is 
typically thought of as a kind of involvement in the 
educational process and it is almost always conceptualized 
as having multiple dimensions with the most common being 
behavioral, affective, and cognitive (Appleton, 2008; 
Fredericks et al., 2004). Accordingly, students are said to be 
engaged when they behave in certain ways (attend class, 
participate in class-related activities, etc.); when they have 
certain sorts of feelings (of belonging, of enjoyment, of 
identification with academic pursuits, etc.); and when they 
have certain types of beliefs and thoughts (perceiving that 
subject matter is interesting, worthwhile, etc.) (Jimerson, 
Campos & Greif, 2003). Moreover, engagement is usually 
conceived as having a positive association with academic 
achievement, including learning outcomes as well as 
completion, persistence, time to degree, etc. (Appleton et al., 
2006; Christenson & Thurlow, 2004). 

Social context, as we think of it, differs from engagement 
in its essentially interpersonal nature. Depending on the 
theory, student engagement may or may not include a social 
component, whereas social context consists of the network 
of inter-relationships in the classroom, between instructors 
and students as well as among students themselves. In fact, 
social context may help to determine how engaged students 
are in the educational process, and if so, it should be 
conceptualized and measured separately. This observation 
becomes particularly relevant when we conceive of how 
classroom space affects learning. A classroom that facilitates 
social interaction may affect student engagement, and 
possibly learning outcomes, differently than a traditional 
classroom that does not invite the same social connections. 

Methodology 

Item generation 

In preparing to design items, we reviewed our existing 
data on social interaction in the classroom. We consulted 
transcripts of interviews with instructors, recordings of 
student focus groups, formal observation logs, and 
responses to hundreds of open-ended survey questions from 
both instructors and students. Additionally, we reviewed 


the educational literature on working alliances and social 
context. From this initial review we developed 63 items that 
mapped to constructs we identified in our data and in the 
literature. Our aim was to capture the spectrum of social 
interactions among students and instructors as it might 
manifest both inside and outside the classroom. All of the 
items conformed to guidelines for writing effective 
questionnaire items that minimize measurement error 
(Dillman, 2014). 

We sought content validity by reviewing items with four 
researchers and conducting three think-aloud interviews 
with students. As a result of these procedures, we 
eliminated items that were deemed not to express an aspect 
of social relations. For instance, the item am responsible 
for my own learning" only implicitly involves a social 
connection and was thus removed. We also revised 
ambiguous wording when it arose in the think-aloud. For 
example, students considered the item “I know the 
instructor as a person" to indicate a high level of connection 
that was rarely if ever achieved; however, students felt that 
they more frequently became "acquainted" with an 
instructor, so we revised that item to reflect that nuance. 
Additionally, because our plan was to make comparisons 
between traditional classroom seating arrangements and the 
more flexible configurations of the ALCs, we eliminated any 
mention of "groups" or "partners" and opted for references 
to students "sitting near me." This change makes the SCALE 
survey agnostic with respect to the type of learning 
environment in which it is implemented and ideal for 
making comparisons between room types. 

Pilot test: Exploratory factor analysis 

The item-generation process resulted in a survey that 
contained 31 items measured on an agree/disagree scale; we 
administered this survey to 842 introductory science 
students at the University of Minnesota in 2015. To explore 
the underlying dimensions in the data, we conducted an 
exploratory factor analysis using principal components 
analysis (PCA). We chose PC A because it attempts to explain 
as much of the total variance in an observed data set as 
possible, using a set of latent factors. We chose an oblique 
rotation (oblimin with Kaiser normalization) because we 
expected the factors to be correlated. 

Following recommendations from Costello et al. (2005), we 
used Kaiser's criterion (Eigenvalues greater than one), the 
shape of the scree plot, and substantive interpretability to 
determine how many components to retain from the 
analysis. The initial PCA yielded 5 factors using Kaiser's 
criterion, but the fifth of these factors contained just two 
items and had an Eigenvalue of only 1.030, while the scree 
plot showed a distinct drop in plotted Eigenvalues between 
factors 4 and 5. These two items were therefore removed 
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from the analysis. Furthermore, two additional items were 
removed because their factor loadings were below .40 and 
hence were too low for practical significance (Netemeyer, 
Bearden, & Sharma, 2003). 

A second PCA with oblimin rotation was conducted on 
the remaining 27 items. The Kaiser-Meyer-Olkin measure of 
sampling adequacy was acceptable (KMO = .929), Bartlett's 
test of sphericity was significant (x 2 = 10259.86, p < .000), and 
all criteria recommended retaining 4 factors which together 
explained 58.21% of variance in the data set. Some 
researchers suggest that PCA and factor-analytic techniques 
like principal axis factoring (PAF) will yield similar results if 
the data set does not contain excessive measurement error 
(Costello, 2005; Teo, 2013), therefore another analysis was 
run on the remaining 27 items, using PAF with oblimin 
rotation. This analysis produced a 4-factor solution nearly 
identical to the one obtained from the PCA, indicating low 
measurement error in the data. 

The 4-factor solution divides into two broad categories: 
two factors describe student interactions with each other and 
2 factors outline the connection between the student and 
instructor. Factor 1 (Student-Student General Relations) 
comprised 10 items pertaining to whether the respondent 
knew other students in the class and had learned from them. 
Factor 2 (Student-Instructor Formal Relations) contained 5 
items that focused on more formal aspects of class, like 
asking questions during class, taking tests or handling 
assignments Five items made up factor 3 (Student-Instructor 
Informal Relations) and had to do with mutual acquaintance 
between students and instructor. Finally, the 7 items in factor 


4 (Student as Instructor) all pertained to the respondent's 
playing the role of instructor with respect to his or her fellow 
students. 

All of the factors had reliability coefficients greater than .7, 
and none could attain greater reliability by removing any of 
their constituent items. Correlations between the factors 
were positive, significant at the p < .01 level, and only 
moderate in size (ranging from .208 to .526), indicating that 
the 4 factors are sufficiently independent to contribute 
separately to the overall structure of the data. 

Validation: Confirmatory factor analysis 

In 2016, we administered the 27-item, 4-factor SCALE 
survey to 344 introductory science students at the University 
of Minnesota and subjected the resulting data set to 
confirmatory factor analysis using AMOS 24.0 in order to 
validate the factor solution that had emerged from the PCA. 
We began by examining the Mardia's skewness and kurtosis 
coefficient to determine whether our data were multivariate 
normally distributed and hence appropriate for structural 
equation modeling. Raykov & Markoulides (2008) suggest 
that the Mardia's coefficient should be less than (# of 
indicators)*!# of indicators + 2); in our data set, and this value 
= 27*29 = 783. The Mardia's coefficient for our data was 
123.962, substantially less than 783, so we proceeded with 
the analysis. 

(Table 1 shows the four-factor solution along with 
reliability coefficients for each factor; the full list of items 
retained at this stage, along with factor loadings for each 
item, is provided in Appendix 1.) 


Table 1. Four-factor solution for measuring social context 


Factor 1: Student- 

Student General 

Relations 

Factor 2: Student- 

Instructor Formal 

Relations 

Factor 3: Student- 

Instructor Informal 

Relations 

Factor 4: Student as 

Instructor 

Reliability (a) 

.915 

.727 

.847 

.835 

Eigenvalue 

8.82 

3.26 

2.25 

1.39 

Percent variance 
explained 

32.66 

12.06 

8.34 

5.15 

Items 

Ql, Q8, Q9, Q14, Q15, 
Q21, Q24, Q25, Q26, 

Q27 

Q3, Q6, Qll, Q17, Q22 

Q5, Q10, Q12, Q18, Q20 

Q2, Q4, Q7, Q13, Q16, 
Q19, Q23 

Sample item 

Q21:1 am acquainted 
with the students sitting 
near me in class. 

Qll: My instructor wants 
me to do well on the tests 
and assignments in this 
class. 

Q20: I've spoken 
informally with the 
instructor before, 
during, or after class. 

Q4: The people sitting 
near me have learned 
something from me 
this semester. 
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Because the four factors had been found to be correlated 
in the exploratory stage, the latent variables representing 
those factors were assumed to covary in the confirmatory 
model. Further, the SCALE survey items were examined and 
the items that were substantively related to one another were 
also allowed to covary in the confirmatory model (Blunch, 
2013). Each indicator variable was associated with exactly 
one latent factor, and indicator variables associated with one 
latent factor were not permitted to covary with indicators 
associated with a different latent factor. 

We report here several measures of model fit. Because the 
X 2 statistic associated with structural equation models is too 
strongly influenced by sample size, we report the normed 
X 2 /df ratio, where the range of desirable ratios is between 2 
and 5 (Carmines & Mclver, 1981; Marsh & Hocevar, 1985), 
with lower ratios indicating better model fit. We also report 
the absolute index of fit RMSEA (good fit indicated by values 
close to .05), and two incremental fit indices, namely the 
comparative fit index CFI, and the Tucker-Lewis Index, with 
desirable values above .90 (Hu & Bentler, 1999). 

The initial CFA indicated moderate model fit (x 2 705.028, 
X 2 /df ratio 2.335, CFI .913, TLI .899, RMSEA .062). One item 
(Q10) was removed because it was implicated in many of the 
modification indices produced by AMOS and was 
associated with several standardized residual covariances 
above 3. (This item also had a relatively low factor loading 
and cross-loaded substantially on more than one factor in the 
exploratory stage.) After removing this item, all indicators of 
model fit improved: x 2 583.642, y}jdf ratio 2.11, CFI .929, TLI 
.917, RMSEA .057. 

We compared two alternative models to our 4-factor, 26- 
item model: first, an uncorrelated or null model, and second, 
a 2-factor model that conceptualized all 26 items into two 
dimensions, namely a student-student factor and a student- 
instructor factor. The fit indices for all three models are 
shown in Table 2. The results suggest that the 4-factor model 
fits the data best. 


Table 2. Fit indices for three alternative models 

Model 

X 2 

X 2 /df 

CFI 

TLI 

RMSEA 

Uncorrelated 

1008.790 

3.172 

.851 

.836 

.079 

Two-factor 

1330.469 

4.334 

.778 

.747 

.099 

Four-factor 

583.642 

2.11 

.929 

.917 

.057 


The graphical representation of the 4-factor SCALE 
instrument is shown in Figure 1, in the form of a path 
diagram. 

Discussion 

The purpose of this study was to examine the 
psychometric properties of an instrument designed to 
measure different aspects of classroom social context. The 


results provided support for the validity and reliability of 
the 26-item SCALE survey, which conceives of social context 
as consisting of four distinct dimensions, two having to do 
with student-student relations and two with student- 
instructor relations. Little cross-loading of items was found 
in the exploratory stage. The dimensions of social context 
were correlated positively and significantly but only 
moderately, suggesting that the dimensions measure 
distinct aspects of social context. The SCALE survey can be 
used by instructors teaching in different types of learning 
spaces to measure the ways in which space affects the social 
aspects of the teaching-learning process. 


Figure 1. Path model of the 4-factor SCALE instrument 

This study was based on data derived from a relatively 
large sample of college students (over 800 in the exploratory 
stage and nearly 350 in the confirmation stage), but its 
conclusions are limited by the fact that our participant 
population was relatively homogeneous in age and ethnicity 
all of whom were enrolled at a large, urban, Midwestern 
university. Further validation of the SCALE survey could 
extend its application to graduate students, to smaller 
colleges, to more diverse student populations, etc. 
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In an attempt to map the connections between the four 
dimensions of social context and the learning outcomes that 
students achieve research currently underway at our 
university will apply the SCALE survey to live classroom 
contexts. Through this new study we hope to answer two 
questions that are crucial to establishing the importance of 
social context in the study of new learning spaces: (1) Is 
social context as measured by the SCALE survey different in 
different types of learning spaces (e.g., ALCs vs lecture halls) 
(2) Does social context matter to student learning. If the 
answer to both of these questions is yes, then social context 
may be a plausible mechanism underlying the impact that 
different learning spaces have on teaching and learning. Not 
only would this result expand our theoretical understanding 
of how learning spaces work, but from a practical point of 
view, it would also suggest ways for instructors to support 
their students' learning by working to improve the social 
context in their classrooms. 
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Appendices 

Appendix 1: Full list of SCALE items with factor loadings 

(All questions answered on a 5-point scale from "Strongly Agree" to "Strongly Disagree") 


Item 

Factor loading 

Factor 

Ql: I've learned something from my classmates. 

.750 


Q8: The students sitting near me rely on each other for 
help in learning class material. 

.751 


Q9: In general, the people sitting near me in class work 
well together on class assignments, questions, etc. 

.821 


Q14:1 know something personal about the people 
sitting near me in class. 

.747 


Q15:1 feel comfortable asking for help from my 
classmates. 

.679 

Factor 1: Student-Student General 

Relations 

Q21:1 am acquainted with the students sitting near me 
in class. 

.838 


Q24: During class, I often have a chance to discuss 
material with some of my classmates. 

.718 


Q25: The students sitting near me respect my opinions. 

.611 


Q26: Other students pointed out a helpful resource. 

.663 


Q27: Other students explained a concept to me. 

.866 


Q3: The material covered by the tests and assignments 
in this class was presented and discussed in class or 
online. 

.536 


Q6: My instructor makes class enjoyable. 

.579 


Qll: My instructor wants me to do well on the tests and 
assignments in this class. 

.498 

Factor 2: Student-Instructor Formal 

Relations 

Q17: Sometimes I feel like my instructor and I are on 
opposing teams in this class. 

-.719 


Q22: My instructor encourages questions and 
comments from students. 

.638 



Journal of Learning Spaces, 6(3), 2017. 


40 




















MEASURING SOCIAL RELATIONS IN NEW CLASSROOM SPACES 


Q5: The instructor knows my name. 

.775 


Q10: The instructor seems to care about me. (Question 
removed in the CFA stage) 

.489 

Factor 3: Student-Instructor 

Informal Relations 

Q12: The instructor is acquainted with me. 

.864 

Q18:1 am acquainted with the instructor. 

.835 


Q20: I've spoken informally with the instructor before, 
during, or after class. 

.771 


Q2:1 can explain my ideas in specific terms. 

.696 


Q4: The people sitting near me have learned something 
from me this semester. 

.560 


Q7:1 can clearly explain new concepts I've learned to 
others in class. 

714 


Q13:1 can persuade my classmates why my ideas are 
relevant to the problems we encounter in this class. 

.484 

Factor 4: Student as Instructor 

Q16:1 can use the terminology in this class correctly. 

.708 


Q19:1 can explain my thought process from start to 
finish to others in class. 

.818 


Q23:1 can help others in this class learn. 

.600 


The instructor is often amusing. 



I know at least one thing about the instructor that is 
unrelated to the course. 


Questions removed from the 
instrument in the EFA stage 

My instructor is approachable. 



I respect my instructor. 
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